c:- 



Europdlsches Patentamt 
European Patent Office 
Office europ6en des brevets 



@ Publication number : 0 540 329 A2 


® 


EUROPEAN PATENT APPLICATION 


(21) Applk:ation number: 92309915.4 
(g) Date of filing : 29.10.92 


@ int ci.^ G11B 20/10, G11B 27/30 


@) Priority: 30.10.91 R 915113 

@ Date of publication of application : 
05.05.93 Bulletin 93/18 

@ Designated Contracting States : 
DE ES FR GB IT NL SE 

Applk:ant : SALON TELEVISIOTEHDAS OY 
Salorankatu 5-7 
SF-24101 Sato (Fl) 


@ Inventor : Kuusama, Juha 
Fysilkanpoiku 2 B 22 
SF-33720 Tampere (Fl) 

@ Representative : Singleton, Jeffrey 

Eric Potter & Ciarkson St Mary's Court St 
Mary's Gate 

Nottingham NGi 1LE (GB) 


@ Method for storing a multichannel audio signal on a compact disc. 

@ On a known compact disc, in which the information in digital form consists of iefl-<:hannel and 
right-channel audio samples and of user data, it is possible to store a multichannel, for example surround 
sound, audio signal so that the multichannel audio signal is matrized into a first audio signal (L) and a 
second audio signal (R), for each of which there is calculated a masking threshold below which sounds 
are not audible to the human ear. The portion below the masking tiireshoid is substituted by bits of a 
multichannel audio signal converted into a bit stream. All the control data required for extracting tiie 
signals containing multichannel information from the first and second audio signals are recorded as part 
of the user data of the CD, in its subcode words. 
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The invention relates to a method by which it is 
also possible to record a multichannel sound on a CD 
(Compact Disc) on which a two-channel stereo sound 
and the control data required by the coder of the re- 
producing equipment are recorded in a known man- 
ner. The invention also relates to a disc on which a 
multichannel sound has been recorded in accordance 
with the invention. 

Multichannel sound has currently become conrv 
mon as the sound in motion pictures, because it pro- 
vides a listening enjoyment greater than does the 
two-channel stereo sound. However, the completed 
sound recordings on CDs currently available for con- 
sumers to purchase are conventional stereophonic 
recordings. It is to be expected that the requirement 
for multichannel sound reproduction from CDs will be- 
come more common in the future. In that case, of 
course, the first problem to be solved is how a multi- 
channel sound can be recorded on a CD. The record- 
ing should be such that it could also be listened to as 
a conventional stereo signal by using present-day re- 
producing equipment. Questions pertaining to the 
transmission and reception of multichannel sound 
have also been discussed In the area of television 
systems. For example, the bandwidth of the audio 
channel in a digital NICAM system does not as such 
suffice for the transmission of a multichannel sound 
signal, and so a multichannel, e.g. four-channel, 
sound must be coded in some manner so that it will 
be suitable for transmission on the transmission path 
of a two-channel stereo signal. Furthernrwre, the cod- 
ing must be done so that the received multichannel 
sound signal can as such be listened to as a two- 
channel stereo signal by using present-day receivers. 
The problem concerning television systems has been 
solved in Finnish patent application Fl-915114filed si- 
multaneously with the present application, the appli- 
cant being Salon Televisiotehdas Oy. Therein, a 
method is described by which a multichannel sound 
can be coded to render it suitable for transmission on 
a channel intended for the transmission of a two- 
channel sound. The coding method presented In con- 
ference publication Proc. ICASSP 90, Alberquerque, 
New Mexico, April 3-6, 1990. pp. 1097-1100, W.R.Th. 
ten Kate. LM. van de Kerkhof and F.F.M. Zijderveld: 
Digital Audio Carrying Extra Information is used in 
the said application. The coding method has been de- 
veloped by Philips. 

The coding method utilizes a factor characteristic 
of human hearing. I.e. the masking effect The mask- 
ing effect means that to any audio signal it is possible 
to add another, weaker signal which is not at all audi- 
ble to the ear owing to the masking effect The mask- 
ing effect is a psychoacoustic effect in which the au- 
ditory threshold shifts upwards when there are 
sounds lower than others present The masking ef- 
fect works best with sounds the spectral components 
of which are close to the components of the masking 


sound. Frequency masking weakens faster when a 
shift is made to lower sounds. This also applies on the 
time scale: the masking effect Is greatest with sounds 
which are simultaneous. The existence of the mask- 
5 ing effect can be exploited by adding to an audio sig- 
nal signals which are below the auditory threshold. In 
principle this is done by sampling the audio signal and 
by substituting other information for those bits which 
are not audible to the human ear. The information is 
10 thus substituted for the less significant bits of the dig- 
ital-form sample. When such a signal is reproduced, 
the human ear will not at all hear the added signal, for 
the actual signal intended for hearing will mask It The 
masking capacity of the human ear thus determines 
IS how many less significant bits can be substituted 
without the substitution being yet audible. 

In the above-mentioned patent application Fl- 
915114, the information produced by the coder of the 
prior known Philips system is exploited. Such infbr- 
20 mation includes information regarding the data 
mode, information relating to quantization, and infor- 
mation relating to dematrization. According to the in- 
vention disclosed in the said application, this infor- 
mation is transmitted to the receiver on a separate 
25 side channel, simultaneously with the audio signals; 
controlled by the side-channel information the receiv- 
er will be capable of processing the stereo signal it re- 
ceives and of converting the signal, for example, into 
a four-channel stereo signal. Briefly, part of the infor- 
30 nrtation of the multichannel sound is hidden in the 
sound of the left and right stereo channels by taking 
advantage of the masking effect of the human ear. 
The rest of the multichannel sound is transmitted on 
the separate side channel on which the information 
35 fordecoding is transmitted. The decoder of the receiv- 
er thus functions under the control of the coder of the 
transmitter, as a slave decoder, and decodes the 
sound back into a multichannel sound. Without mul- 
tichannel sound decoding it is, however, possible to 
40 receive the sound transmitted on the stereo channel 
and, owing to the masking effect, to reproduce it as a 
normal stereo sound without the listener hearing the 
hidden sound. The coder described in the Ft applica- 
tion, which may for its essential parts be similar to the 
45 Philips coder described in the said conference public- 
ation Prdc. ICASSP 90, combines the incoming nmjl- 
tichannel audio signal to form a combined stereo sig- 
nal and, by making use of the masking effect, hides 
a data signal In theformed two-channel stereo signal. 
50 Information regarding the data mode, quantization 
and matrfczation is obtained from the coder. The quan- 
tization information indicates the quantizatbn steps 
and number of bits of the masking signal and of the 
signal to be masked (hidden), as well as the masking 
55 threshold which has been calculated for the audio sig- 
nal by time intervals. The matrization information in- 
dicates how the original multichannel audio signal 
was downmixed. Briefly, all the information by means 
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of which the decoding can be carried out can be ob- 
tained from the coder. The combined stereo signal ob- 
tained from the coder. In which data have been "hid- 
den**, is adapted to the audio channel used, for exam- 
ple the NiCAM format, for transmission on the radio 
path. The above-listed information, necessary for de- 
coding, is transmitted simultaneously on a separate 
low-speed digital channel. If the data to be hidden in 
the audio channel do not at a given point of time fit in 
the audio channel - the masking capacity of the audio 
signal *s not sufficient - these data can be transmit- 
ted on the said separate data channel; the informa- 
tion transmitted on this channel can be called side in- 
formation since it is transmitted on the side of the ac- 
tual audio channel. 

In the receiver the decoder receives the signal of 
the audio channel and the side information of the 
data channel, and controlled by the decoding Infor- 
matbn transmitted in the latter, it will be able to de- 
code the signal of the audio channel and to extract 
the data hidden in it Controlled by the matrization in- 
formation it will further be able to form, for example, 
a multichannel audio signal. The data channel used 
for the transmission of the side information may be 
slow, and the data transmitted on it can be easily conrv 
pressed and protected. The method is especially suit- 
i able for being used for transmitting a multichannel 
sound (surround sound) in a high-definition television 
system. 

it was mentioned at the beginning of this specifi- 
cation that the CDs currently on sale have two-chan- 
nel stereo sound. The problem how a multichannel 
sound can be recorded on a CD can be solved by us- 
ing the principle of the said Fi application. Thus, when 
reproduced by present-day equipment, the recording 
will sound quite tike a conventional two-channel ster- 
eo sound, but when the reproducing equipment is pro- 
vided with a special decoder, the recording can be re- 
produced as the original multichannel sound, so-cal- 
led surround sound. 

This recording is made according to the method 
of Claim 1. 

The invention is based on the realization that 
since, as is known, also data information necessary 
for decoding is always recorded on a CD in addition 
to the stereo sound, the data required by the repro- 
duction of a multichannel signal can be included 
among this data information. This is possible, since 
the space reserved on a CD for data information al- 
ready has reservation for the recording of extra data 
by providing redundant data capacity for this purpose. 
More precisely, this means that the side-channel in- 
formation defined in patent application FI-915114 is 
recorded in the redundant data capacity in the data 
blocks of the CD, and sound of the other channels is 
hidden on the two audio channels of the disc by ex- 
ploiting the masking effect. When the sound is being 
reproduced as a multichannel sound, the recording is 


dematrized by making use of the side-channel infor- 
matton and the sound information hidden in the two 
audio channels. The side-channel informatton cannot 
be used with conventional CD decoders, and so the 
5 reproduced sound will be a two-channel stereo sound 
which, owing to the masking effect, will sound like the 
original one in spite of the sound hidden in it On the 
other hand, when a coder controlled by the side- 
channel information is used, a multichannel sound 
10 will be obtained. 

The invention is described with reference to the 
accompanying figures, in which 

Figure 1 depicts the structure of one data frame 

of a CD, 

15 Figure 2 depicts the general principle of coding a 

multichannel sound and of hiding it in the stereo 
channel, and 

Figure 3 depicts the decoding of a multichannel 
sound recorded on a CD. 

20 The information on a CD is made up of succes- 

sive data frames such as shown in Figure 1. Each 
frame includes 33 symbols (bytes), which are preced- 
ed by a preamble, and one symbol comprises 8 data 
bits. The first symbol of the frame Is used for the sub- 

25 code, and 24 of the remaining 32 bytes represent au- 
dio samples and eight bytes are used for error correc- 
tion purposes to correct tnjrst errors and random er- 
rors. The subcode as a whole is rather extensive, and 
only part of it is contained in one frame. The entire 

30 subcode Is placed in 98 successive frames. Relative 
to the audio signal the subcode is essentially an aux- 
iliary data stream which has been placed among the 
audio samples. Its function is, among other things, to 
assist in finding the starting points of the different 

35 musical pieces on the disc, in locating them on the 
disc, in cataloging the durations of the pieces, and in 
the cumulative time counting of the disc. Further- 
more, it conveys information on the pre-emphasis 
used in the recording so that the disc player can au- 

40 tomatically select the correct method of de-empha- 
sis. The format of the subcode is standardized. The 
reproducing equipment will find It on the disc so that 
each frame begins with a synchronization pattern. 
The disc player recognizes the pattern and uses it for 

45 incrementing the frame counter. The different sub- 
codes are distinguished so that there is a synchroni- 
zation pattern also in the subcode area, in which case 
two successive synchronization patterns separate 
the different subcode blocks from each other. Since 

50 the subcodes can be thus recognized, they can be 
extracted from the frames so that the audio samples 
are directed to their own audio sample processing 
branch. It was stated above that the subcode is 
placed in 98 frames. The disc player extracts the sut>- 

33 codes and stores them in the RAiV^ memory. When 98 
eight-bit subcodes have been extracted, the proces- 
sor of the player will take this as eight 9&-bit code 
words. Thus each block of 98 frames has 8 such code 
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words, but at present only one of these 96-bit words 
has significance, I.e. it is in use, and the rest of the 
seven code words which fit in the storing capacity of 
the block are reserved for future use. Thus the free 
capacity for storing the side information according to 
the invention amounts to seven 96-bit words every 
13.3rd nnillisecond, i.e. 50526 bits/second. This ca- 
pacity IS usable for storing the side-channet informa- 
tion. 

According to the invention, in the frames other 
than the bytes of the audio samples there is placed 
side-channel information which, in addition to any au- 
dio samples, contains all the information required for 
controlling the decoder of the reproducing equipment. 
These bytes, in which the side-channel information is 
recorded, constitute, in accordance with what Is stat- 
ed above, the free capacity of the subcode, i.e. 7 
words of 96 bits per each block of 98 frames. The re- 
cording speed is, if the entire free capacity were used, 
50.526 kbit/s. Any other multichannel information is 
hidden in the audio samples themselves. 

When multichannel material is being recorded on 
a CD, the procedure is in accordance with Figure 2. 
as follows: The multichannel sound is converted in 
one way or another into a bit stream by means of the 
multichannel sound coder. The coder may function 
according to any prior known method, for example Nl- 
CAM, MUSICAM. PCM. Let us assume, for example, 
as follows: The original music recording is in exis- 
tence in two forms, I.e. as a stereo sound L. R and as 
a multichannel sound A1, A2. A3.. .An. The producer 
of the performance has, of course, in some manner 
made the stereo sound from the multichannel sound. 
The multichannel sounds A1...An are thus coded as 
a bit stream in accordance with Figure 2, in the mul- 
tichannel sound coder 1. The data of this bit stream 
are hidden in the stereo sound in a known manner in 
block 2. A new stereo pair L', R' is obtained. In the 
manner described in application FI-915114, also a 
side channel is formed the information of which con- 
tains the necessary data relating to the decoding of 
the multichannel sound, i.e. the decoder control data. 
That portion of the decoder bit stream which cannot 
be hidden In the stereo pair L, R Is also transmitted 
on the side channel. 

When a multichannel sound is being recorded on 
a CD, each channel L' and R' containing hidden infor- 
mation is recorded on its own audio channel. The con- 
trol channel information is recorded In the place of the 
free subcode words of the user data on the disc. In re- 
cording on the disc, already existing equipment can 
thus mostly be used. 

When the CD is being played on a player, the pro- 
cedure is in accordance with Figure 3. As at present, 
the subcode data and the audio-channel data are 
read from the disc. If present-day players are used for 
reproducing the sound, stereo channels L' and R' are 
obtained for reproduction. Owing to the masking ef- 


fect they will sound to the listener quite the same as 
the original stereo channels L and R. When the player 
is equipped with an auxiliary device for decoding the 
multichannel sound, the audio signals L' and R' and 
5 the side-channel information read from the subcode 
data are led to the demasking block 3, where the in- 
formation hidden in the audio signals L', R' is extract- 
ed from them. The data extraction is carried out under 
the control of the side-channel information read from 
10 the subcode data. Thus a bit stream is obtained which 
is the same as the bit stream coming from the coder 
of Figure 2. This bit stream is applied to the nmjitichan- 
nel sound decoder 4 which, according to the control 
data contained in the bit stream, will decode the orig- 
15 inal multichannel audio signal. The decoder is thus, 
for example, a NICAM. MUSICAM or PCM decoder, 
depending on the coder used. 

If the multichannel sound data hidden in the sig- 
nal of the right channel R and the left channel L are 
20 not removed, the signal is not quite the same as in the 
original, but owing to the masking effect, the listener 
will not notice any difference. Since the format of a 
multichannel sound may be made identical to the for- 
mat of the audio signal to be recorded on the CD and. 
25 by a suitable arrangement, identical to the format of 
the data to be recorded on the CD, a CD recording 
made by the method of the invention can be used for 
reproducing a two-channel audio signal with conven- 
tional playere or for reproducing a multichannel ster- 
30 eo signal (surround sound) when using in the player 
a decoder equipped with suitable additional circuits. 

The CD according to the invention has many ad- 
vantages. It is completely compatible with existing 
CD equipment The disc can be manufactured and it 
35 can be played using current equipment, without any 
additional auxiliary devices. If the coding/decoding 
method utilizing the skJe channel presented by the 
applicant is used in a HDTV television system, the CD 
recording will be compatible with the HDTV audio ma- 
40 terial at the bit level. 


Claims 

45 1. A method for storing a multichannel (A1 , A2...An) 
audio signal on a compact disc, wherein the infor- 
mation in digital form is made up of the audio 
samples of the left channel and the right channel 
andof the user data which the reproducing equip- 

50 ment will need for arranging the audio samples 

into a stereo signal, and in which method the mul- 
tichannel audio signal (A1. A2..An) is coded so 
that a bit stream is obtained, a f iret audio signal 
(L) and a second audio signal (R) are formed from 

55 the multichannel audio signal, there being calcu- 

lated for each of them a masking threshold below 
which sounds are not audible to the human ear, 
and the bits of the said bit stream are substituted 
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for those bits in the first and second audio signals 
which remain below the masking threshold, 
whereby a converted first audio signal (L') and a 
converted second audio signal (R') are obtained, 
characteiizsd in that 

- all the control data required for extracting 
the bits of the said bit stream from the first 
and second audio signals are collected, 

- the converted first audio signal (L') and the 
converted second audio signal (R') are stor- 
ed as the audio samples of the left channel 
and the right channel of the CD, 

- the collected control data are stored as part 
of the user data of the CD, in its subcode 
words. 


10 


nels (L' and R') and for reconstructing the 
original multichannel audio signal are incor- 
porated into the user data subcoding. 

A compact disc according to Claim 6, character- 
ized In that the user data subcoding also includes 
that portion of the bit sfream which it has not 
been possible to place below the masking thresh- 
old of the audio samples of the first channel (L) 
and the second channel (R). 


IS 


2. A method according to Claim 1. characterized in 
that also that part of the bit stream which cannot 
be combined with the first and the second audio 
signals is combined with the control data. 


20 


A method according to Claim 2. characterized In 
that the format of the multichannel audio signal 
is 3/2, which means that it has a left channel (L), 
a right channel (R) and a center channel (C), as 
well as two surround channels (SR, SL) on the 
sides. 


25 


4. A method according to Claims 1 and 3, charac- 
terized In that the first audio signal contains the 30 
left channel (L) Information and the second audio 
signal contains the right channel (R) Information. 

5. A compact disc the digital-form information on 
which is made up of audio samples of the left 35 
channel and the right channel and of user data re- 
quired by the reproducing equipment for arrang- 
ing the audio samples into an audio signal, char- 
acterized in that 

- the audio samples of the first channel (L') 40 
are made up of the first-channel (L) sanr>- 

ples of a stereo sound formed from a multi- 
channel audio signal (A1, A2...An), in which 
samples those bits which are below the 
masking threshold have been substituted 45 
by bits of a multichannel audio signal coded 
into a bit stream 

- tlie audio samples of the second channel 
(R') are made up of the second-channel (R) 
samples of a stereo sound formed from a so 
multichannel audio signal (A1, A2...An), In 
which samples those bits which are below 

the masking threshold have been substitut- 
ed by bits of the multichannel audio signal 
coded into a bit stream. 55 

- all the control data required for extracting 
the bits of the bit stream from the audio 
samples of the first and the second chan- 
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@ Method for storing a multichannel audio signal on a compact disc. 


(g) On a known compact disc, in which the infbnmation in digital form consists of left-channel and 
right-channel audio samples and of user data, it is possible to store a multichannel, for example surround 
sound, audio signal so that the multichannel audio signal is matrized into a first audio signal (L) and a 
second audio signal (R), for each of which there is calculated a masking threshold below which sounds 
are not auditrfe to the human ear. The portion below the masking threshold is substituted by bits of a 
multichannel audio signal converted into a bit stream. All the control data required for extracting the 
signals containing multichannel infonmatton from the first and second audio signals are recorded as part 
of the user data of the CD, in its subcode words. 
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